Attractor neural networks storing multiple space representations: a model for 

hippocampal place fields 
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A recurrent neural network model storing multiple spatial maps, or "charts", is analyzed. A 
network of this type has been suggested as a model for the origin of place cells in the hippocampus 
of rodents. The extremely diluted and fully connected limits are studied, and the storage capacity 
and the information capacity are found. The important parameters determining the performance of 
the network are the sparsity of the spatial representations and the degree of connectivity, as found 
already for the storage of individual memory patterns in the general theory of auto-associative 
networks. 

Such results suggest a quantitative parallel between theories of hippocampal function in different 
animal species, such as primates (episodic memory) and rodents (memory for space). 
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I. INTRODUCTION 

Knowledge of space is one of the main objects of com- 
putation by the brain. It includes the organization of 
information of many kinds and many origins (memory, 
the different sensory channels, and so on) into mental 
constructs, i.e. maps, which retain a geometrical nature 
and correspond to our perception of the outer world. 

Every animal species appears to have specialized sys- 
tems for spatial knowledge in some region of the brain, 
more or less well developed and capable of performing 
sophisticated computations. For rodents there is a large 
amount of experimental evidence that the hippocampus, 
a brain region functionally situated at the end of all the 
sensory streams, is involved in spatial processing. Many 
hippocampal cells exhibit place related firing, that is, 
they fire when the animal is in a given restricted region of 
the environment (the "place field" ) , so that they contain 
a representation of the position of the animal in space. 

The hippocampus, one of the most widely studied 
brain structures, shares the same gross anatomical fea- 
tures across mammalian species, nevertheless it is known 
to have different functional correlates for example in pri- 
mates and humans (where it is believed to be involved 
in episodic memory, roughly, memory of events) and in 
rodents, in which it is mainly associated to spatial rep- 
resentation. 

One relevant feature of the hippocampus which is 
maintained across species is a region, named CAS, char- 
acterized by massive intrinsic recurrent connections. It 
was appealing for many theorists to model this region 
as an auto-associative memory storing information in its 
intrinsic synaptic structure, information which can be re- 
trieved from small cues by means of an attractor dynam- 
ics, and which is represented in the form of activity con- 
figurations. 

Within the episodic memory framework, each attrac- 
tor configuration of activity is the internal representation 
of some memory item. The CAS auto-associative net- 



work can be seen as the heart of the hippocampal system, 
containing the very complex, inter-modal representations 
peculiar to episodic memory. Auto-associative, or attrac- 
tor neural networks have been extensively studied with 
the tools of statistical physics Efficiency measures, 
like the number of storable memory items or the quality 
of retrieval, can be computed for any appropriately de- 
fined formal model. An intensive effort was performed 
to embed in this idealized models more and more ele- 
ments of biological realism, trying to capture the relevant 
anatomical and functional features affecting the perfor- 
mance of the network. It was then found that a theory of 
the hippocampus (or, more precisely, CAS) as an auto- 
associative network has as its fundamental parameters 
the degree of intrinsic connectivity, i.e. the average num- 
ber of units which send connections to a given unit, and 
the sparsity of the representations, roughly the fraction 
of units which are active in one representation. These pa- 
rameters have biological correlates that are measurable 
with anatomical and neurophysiological techniques. 

Spatial processing, as it is performed by the rodent 
hippocampus, also involves memory of some kind; recent 
experimental evidence support the idea that spatially re- 
lated firing is not driven exclusively by sensory inputs but 
also reflects some internal representation of the explored 
environment. First, place fields are present and stable 
also in the dark, and in condition of deprived sensory ex- 
perience. Second, completely different arrangements of 
place fields are found in different environments or even 
in the same environment in different behavioral condi- 
tions. 

These findings have led to hypothesize that CAS (like, 
perhaps, other brain regions with dense connectivity) 
stores "charts" , representations of environments in the 
form of abstract manifolds, on which each neuron cor- 
responds to a point, that is the center of its place field. 
Place fields arise as a result of an attractor dynamics, 
whose stable states are "activity peaks" centered, in the 
chart space, at the animal's position. It is important to 
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note that the localization of each neuron on a chart does 
not appear to be related to its physical location in the 
neural tissue. 

The positions of place fields arc encoded by the recur- 
rent connections, and it is possible to store many differ- 
ent charts in the same synaptic structure, just as many 
different patterns are stored in a Hopfield net, for ex- 
ample, and different activity peaks can be successively 
evoked by appropriate inputs, just as it happens with 
auto-associative memories. 

It is interesting to address the issue of whether an 
episodic memory network and the "spatial multi-chart" 
memory network share the same functional contraints, 
so that a biological brain module capable of performing 
one of the tasks is also adequate for the other. Here 
we present a statistical mechanics analysis of the multi- 
chart network, focusing on the parallel with autoassocia- 
tive memory in the usual (episodic memory) sense. It is 
found that the performances of these two networks are 
governed by very similar laws, if the parallel between 
them is drawn in the appropriate way. 

In sec. the case of a single attractor chart stored is 
studied, then in sec. Ill the case of multiple stored charts 



is analyzed and the storage capacity is found, first for 
a simplified model and then for a more complex model 
which makes it possible to address the issue of sparsity 
of representations. In sec. ^ the storable information in 
a multi-chart network is calculated, making more precise 
the sense in which such a network is a store of informa- 
tion, and completing the parallel with auto-associative 



memories. 



expressed as 



J., = ^i^(|r,-r,|), 



(2) 



where K is a monotone decreasing function of its argu- 
ment. 

The synaptic input to the i-th cell is therefore given 

by 



Ml 



(3) 



If the number TV of cells is large, and the place fields 
centers (p.f.c.) are homogeneously distributed over the 
environment M (be it one or two-dimensional), we can 
replace the sum over the index j with an integration over 
the coordinates of the p.f.c: 



hir) 



dr'K{\r-r'\)V{r') 



(4) 



M 



Note that the normalization in eq. ^ is chosen in order 
to keep the synaptic input to a given unit fixed when \M\ 
varies and the number of units is kept fixed, that is, the 
density of p.f.c.s A^/|Af | varies (the \M\ factor will then 
compensate for the fewer input units within the range of 
substantial K strength). A fixed-point activity configu- 
ration must have the form 



Vir) = g 



dr'if(|r-r'|)y(r)'-0 



M 



(5) 



II. THE SINGLE MAP NETWORK 



We could write eq. ^ as 



As a first step, we consider the case of a single at- 
tractor map encoded in the synaptic structure, as it was 
proposed in We focus here on the shape and prop- 
erties of the attractor states, as a useful comparison for 
the following treatment of the multiple charts case. 

The neurons are modelled as threshold linear units, 
with firing rate: 

V, = g[h, - 0]+ = g{h, - 0)e{h, - 0) (1) 

i.e. equal to zero if the content of the square brackets is 
negative, h represents the synaptic input current, com- 
ing from other cells in the same module, is a firing 
threshold, which may incorporate the effect of a subtrac- 
tive inhibitory input, common to all the cells, as it will 
be illustrated later on. The connectivity within the mod- 
ule is shaped by the selectivity of the units. If rj is the 
position of the center of the place field of the z-th cell, 
in a manifold M, of size |M|, corresponding to the en- 
vironment, the connection between cells i and j may be 



where i7 is a domain for which there exists a solution of 
eq. Hthat is zero on the boundary. 

If only solutions for which O is a convex domain are 
considered, the fact that V{r) is zero on dfl will ensure 
that units with p.f.c. outside Q are under threshold, 
therefore their activity is zero and solutions of eq. || are 
guaranteed to be solutions of eq. |^. The size and the 
shape of the domain Q in which activity is different from 
zero is determined by eq. ||. As a first remark, we notice 
that it is independent from the value of the threshold 0. 
In fact, if Ve is a solution of (||) with threshold 0, given 
the linearity of eqjl^ within Q, 

0' 
a 

will be a solution of the same equation with 0' instead of 
0, with the same null boundary conditions on fJ. Rescal- 
ing the threshold will then have the effect of rescal- 
ing the activity configuration by the same coefficient. 
This means that subtractive inhibition cannot shape, e.g. 
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shrink or enlarge, this stable configuration, and therefore 
it is not relevant for a good part of the subsequent anal- 
ysis. Some form of inhibition is nevertheless necessary to 
prevent the activity from exploding. Moreover, there are 
fluctuation modes which cannot be controlled by overall 
inhibition as they leave the total average activity con- 
stant. They will be treated in sec. Ill C. It is found that. 



at least in the one dimensional case, these modes do not 
affect stability in the single chart case. 

In absence of an external input, any solution can be at 
most marginally stable, because a translation of the solu- 
tion is again a solution of eq. ||. An external, "symmetry 
breaking" input, taken as small when compared to the 
contribution of recurrent synapses, is therefore implicit 
in the following analysis. 



A. The one-dimensional case 

The case of a recurrent network whose attractors re- 
flect the geometry of a one-dimensional manifold, be- 
sides being a conceptual first step in approaching the 
2-dimensional case, is relevant by itself, for example in 
modeling other brain systems showing direction selectiv- 
ity, e.g. in head direction cells [||J|], and also for place 
fields on one-dimensional environments . 

In this case eq. || reads: 



V{r) 



K{\r ~r'\)V{r') 



V{R) = V{-R) = 0. 



(7) 



For several specific forms of the kernel K it is possible 
to solve explicitly eq. 0, yielding interesting conclusions. 
For example if: 



(see also 



K{\r-r'\) = e-l'-'-'l, 
differentiating eq. twice yields: 
V"{r) = -^''V{r)+ge, 



(8) 
(9) 



where 7 = ^/2g^T. 

Solutions vanishing at — i? and R (and not vanishing 
in 1 — i?, R\), have the form: 



V{r) — ylcos(7r) -f 



9O 



2.9 - 1 ■ 



with 



A 



9O 



(25-l)cos(7i?) 



(10) 



(11) 



The value of R for which (|l^ is a solution of eq. ^ is 
determined by the integral equation itself: for example, 
by evaluating V'{R) or V'{—R) from eq. |^ we get: 



V'{-R) = -V'{R) 



(12) 



Substituting 



so that 



R 



and (|T^) in eq. |l^ we have: 

tan(7i?) — —7 

tan^"'"(— 7) + nTT 
7 



Requiring R to be positive and V{x) to be positive for 
— i? < r < R, leads to choose 



— tan ^(7) -f TT 
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(13) 



note that A> 0. 

R is then a monotone decreasing function of 7, and 
therefore of the gain g. 

This is also true for other forms of the connection ker- 
nel K. As an example, consider the kernel: 



K{r — r') — cos(r — r') 



(14) 



by a similar treatment it is shown that a solution is ob- 
tained with 



R 



1 



(15) 



The kernel 



K{r - r') = 9(1 - |r - r'|)(l - \r - r'\) (16) 



will result in a peak of activity of semi- width 

TT 



R = 



(17) 



Equations of the type (g|) have more solutions in addi- 
tion to the ones considered above, representing a single 
activity peak. For example, if we consider an infinite 
environment, periodic solutions will be present as well, 
representing a row of activity peaks separated by regions 
of zero activity. These solution can be checked to be un- 
stable if we model inhibition as an homogeneous term 
acting on all cells in the same way and depending on the 
average activity. Intuitively, if we perturb the solution by 
infinitesimally displacing one of the peaks, it will tend to 
collapse with the neighbor which has come closer. 
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B. The two-dimensional case 



III. STORING MORE THAN ONE MAP 



To model the place cells network in the hippocampus 
we need to extend this result to a two dimensional envi- 
ronment. The equation for the neural activity will be: 



Vir) = g 



[ dr'K{\r-r'\)V{r')-e 



(18) 



The generalization to 2-D is straightforward if for the 
kernel if(|r — r'|) we consider the one with Fourier trans- 
form is 



^(P) 



(19) 



(the two-dimensional analog of the kernel of cq. that 
is, a kernel resembling the propagator of a Klein-Gordon 
field in Euclidean space. The fact that this kernel is di- 
vergent for (r — r') — > docs not give rise to particular 
problems, since, in the continuum limit of eq. ^, the con- 
tribution to the field h coming from the nearby points 
will stay finite, and in fact two units will be assigned 
p.f.c's so close to each other to yield a overwhelmingly 
high connection only with a small probability. Let us 
look for a solution with circular symmetry such that ac- 
tivity V^(r) is zero outside the circle of radius i?, C{R). If 
we apply the Laplacian operator on both sides of 



V{y) = g [ dr'K{r - r')V^(r') - 9 (20) 



we obtain: 



V'y(r') =-7'F(r')+: 



(21) 



(again, 7^ = 2g — 1), which in polar coordinates reads: 



V"{r) + -V'{r) = -j^V{r)+g6. (22) 



The solution is 

Vir) = AJohr) + 



2g-l 



(23) 



Jo is the Bessel function of order 0. For the solution 
to vanish on the boundary of C{R) one must take: 



(25 - 1) Jo(7i?) 

The other condition that determines R may be found 
by substituting (|2^ ) in eq. |2^. Here again, R{g) is a 
monotone decreasing function. 

As in the one-dimensional case, solutions with a non- 
connected (or even non-convex) support can be seen not 
to be stable. 



Let us imagine now that the p.f.c's for each cell are 
drawn with uniform distribution on the environment 
manifold M, and connections are formed according to 
(^. Several "space representations" may be created by 
drawing again at random the r.p.c. of each cell from the 
same distribution. The connection between each pair of 
cells will then be the sum of a number of terms of the 
form (H) , one for every "space representation" , or "map" , 
or "chart" . With p — aN maps, and the r.p.c. of the 



i-th cell in the fi-th map indicated by x 



p 



I)- 



(24) 



The question that immediately arises is: what is the 
capacity of this network, that is, how many maps can we 
store, so that stable activity configurations, correspond- 
ing to some region in the environment described by one 
map, like the ones described by the solutions of eq. ||, 
are present? The problem resembles the classic attractor 
neural network problem ||l[, with threshold linear units. 
A standard treatment has been developed |^ allowing 
to calculate the capacity of a network of threshold lin- 
ear units with patterns drawn from a given distribution 
and stored by means of a hebbian rule. The treatment is 
very simplified in the extreme dilution limit |^,^. In the 
next sections it will be shown how this treatment can be 
extended to the map case, first for one particular form 
of the kernel K, leading to the solution of the capacity 
problem for a fully connected network; in the following, 
the solution is extended to more general kernels, first in 
the diluted limit, then for the fully connected network. 

Another related question is: how much information is 
the synaptic recurrent structure encoding, and in which 
sense is the synaptic structure a store of information? 
The aim is to develop a full parallel between the multi- 
chart network and autoassociative networks, and if pos- 
sible to characterize the parameters constraining the per- 
formance of this system. 



A. The fully connected network: "dot product" 
kernel 

Let us consider a manifold M with periodic boundary 
conditions, that is, a circle in one dimension and a torus 
in two dimensions. The p.f.c. of a cell can then be 
described by a 2-dimensional unit vector ffi for the one- 
dimensional case and by a pair of unit vectors ffl''^ for 
the two dimensional case. Suppose now that the contri- 
bution from the /z-th map to the connection between cell 
i and cell j is given by: 



Ml 



1), 



(25) 
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so that 



1 , , ^ 

^l=l 1=1 



-Jin) 



1), 



(26) 



Eq. 25 de- 



where d is the dimensionahty. 

p = aN is the number of stored charts, 
scribes an excitatory, very wide spread form for the ker- 
nel (^) (the contribution to the connectivity is zero only 
if the r.p.c.s of the two cells are at the farthest points 
apart, i.e. at 180°). This spread of connectivity would 
lead to configurations of activity that are large in the 
r.p.c. space, that translated in auto-associative memory 
language would be very "unsparse" , i.e. very distributed 
representations. It is therefore plausible that this will 
severely limit the capacity of the net. In any case, the 
form of (^sj) , factorizable in one term depending on fji and 
one term depending on ffj, after incorporating the con- 
stant part in a function b^{x), makes it possible to per- 
form the free energy calculation through Gaussian trans- 
formations as in B . A similar model has been studied in 
with McCuUoch-Pitts neurons. 

A Hamiltonian useful to describe the thermodynamics 
of such a system is 



H 



EEE- 



Hp) . jf(t^) 



(27) 



where B{x) = b{y)dy, and b{x) is a function describ- 
ing an uniform inhibition term depending on the average 
activity in the net. s'*^^) is a symmetry breaking field, 
pointing in a direction in the fj,—th map space. The mean 
field free-energy in the replica symmetric approximation 
can be calculated (the partition function is calculated as 
the trace over a measure that implements the threshold- 
linear transfer function, see [^). The presence of a phase 
with spatially specific activity correlated with one map 
will be signaled by solutions of the mean field equations 
with a non-zero value for the order parameter 



1 



N 

En. 

i=l 



(28) 



which plays the role of the overlap in an auto-associative 
memory. This parameter has the meaning of a popula- 
tion vector [ic[| , that is, the animal position is indicated 
by an average over p.f.c.s of the cells weighted by cells 
activity. 

The set of resulting mean field equations can be re- 
duced to a set of two equations, eqs. A8 and A9, in two 
variables, the "non-specific" signal-to-noise ratio, w, and 
the "specific" , space related signal-to-noise ratio v. The 
details of the calculation are reported in Appendix 

The critical value Oic indicating the storage capa city 
of the network is the maximum value for which eq. 



still admits solutions corresponding to space related ac- 
tivity (non-zero v) and may be found numerically. At 
this value Uc the system undergoes a first-order phase- 
transition towards a state in which no space-related ac- 
tivity is possible. Eq. A£ gives the range of gain values 
for which there exist solutions at a given a < Uc @ . 

In this model there is no possibility for modulating 
the spread of connections in the chart-space. As we an- 
ticipated, the activity configurations that one obtains are 
very wide, with a large fraction of units active at the same 
time. Cells will have very large place fields, covering a 
large part of the environment (of the order of roughly 
one half for the one dimensional case, and roughly one 
quarter for the two-dimensional case). As one would in- 
fer from the analysis of autoassociative memories storing 
patterns, for example binary, these "unsparse" represen- 
tations of space will lead to a very small capacity of the 
net. 

For the model defined on the one-dimensional circle 
the capacity found is etc ~ 0.03. At this value the system 
undergoes a first order transition. As a increases beyond 
etc, X jumps discontinuously from a finite value to zero. 

The capacity for the diluted analogue of this model 
(see Appendix A and section [II B) is given by the 
equation 



(29) 



Remember that in this case p ■ 



acN where c is the con- 
In 



nectivity fraction parameter, see section IIIB. In this 
case ttc ^ 0.25. At ac the transition is second order, 
with the "spatial overlap" x approaching continuously 
zero, verified at least with the precision at which it was 
possible to to solve numerically eq. For the 2-D case, 
storage capacities are ac ~ 0.0008 for the fully connected 
network and a ~ 0.44 for the diluted network. 

To get a larger capacity, and to provide a possible com- 
parison with the experimental data from the hippocam- 
pus, in which the tuning of place fields is generally nar- 
row, we must extend our treatment to more general ker- 
nels, and this will be done in the following two sections. 



B. Generic kernel: extremely diluted limit 

Consider a network in which every threshold-linear 
unit, whose activity is denoted by Vj, senses a field: 



hi — Cij Jij Vj . 



(30) 



where Jij is given by eq. From now on the kernel K 
is defined as 



K{r — r) = K{f — r)—K 
K = Uk{r- f') 



(31) 
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for any r, where ((■■•)) means averaging over f. With 
this notation, whatever the original kernel K, K is the 
subtracted kernel which averages to zero. The manifold 
M is taken with periodic boundary condition (that is a 
circle in one dimension and a torus in the two dimensional 
case). 

Cij is a "dilution matrix" 



^ ,1 with prob. c, 
^3 ~ \ with prob. 1 — c 



(32) 



and Nc/ log N ^ as iV ^ oo. In the thermodynamic 
limit N —f oo the activity of any two neurons Vi and Vj 
will be uncorrelated ||^. A number of charts p = acN 
is stored. Looking for solutions with one "condensed" 
map, that is, solutions in which activity is confined to 
units having p.f.c. for a given chart in a certain neigh- 
borhood, it is possible to write the field hi as the sum 
of two contributions, a "signal", due to the condensed 
map and a "noise" term, pz - z being a random variable 
with Gaussian distribution and variance one - due to all 
the other, uncondensed, maps, namely, in the continuum 
limit, labeling units with the position r^^ of their p.f.c. in 
the condensed map. 



h{f^) = 5 / dr^' K{r^ ~ r'')V{r^') + pz; (33) 
Jm 



the noise will have a variance 

p^^ay\M\mK\f~7^))), 

where 



y 



1 ^ 



N 



(34) 



(35) 



The fixed point equation for the average activity profile 
x^{r) is 



=g J Dz{h{r)-e). (36) 
where again Dz is the Gaussian measure, and 

h{f) = j dfK{f~ r')x^{f) + b{x) ~ pz (37) 



and 



X = r X [r] 



(38) 



is the average overall activity. The average squared ac- 
tivity (entering the noise term) will read: 

y^^^ I W\ f Dz{h{r)-6)\ (39) 



The fixed point equations may be solved introducing 
the rescaled variables 



v(f) 



h{x) - e 
p 

x^(r) 



(40) 
(41) 



The fixed point equation for v{r) is 

v{r) ^ gM [ I dr'K{f-f')v{r')+w] (42) 



where 



J\f{x) = x^{x) + (t{x), 



(43) 



(4>(x) and (j{x) are defined in eq. |A16| and eq. A17) is 
a "smeared threshold linear function" , monotonically in- 
creasing, with 



and 



lim U{x) = 



lim N{x)/x = 1. 

X — > + oc 



In terms of w and v{r), y reads: 
dr 



2 2 

y^pg 



\M 



■M 



dr'K{f-f')v{r')+w] (44) 



where 



Mix) = {l + x^)^{x)+xa{x). (45) 
By substituting eq. ^ in eq. we obtain 



-^g^\M\((K)) I drM{ I dr'K{r-f')v{r)+' 



(46) 



If we can solve eq. ^ and find v{r) as a function of w 
and a solution is found corresponding to a value of a 
given by eq. |4^. To find the critical value of a, we have 
to maximize a over w and g. 

The mathematical solution of eq. ^ is treated in Ap- 
pendix ^ 

With this model, we can modulate the spread of con- 
nections by acting on K{f— f) or alternatively, by vary- 
ing the size of the environment. The results are depicted 
in fig. for the 1-D circular environment and in fig. |^ for 
the 2-D toroidal environment (upper curves). Examples 
of the solutions of eq. ^ arc displayed in fig. |^ for the 
1-D environment and in fig. ^ for the 2-D environment. 

We note that, as the environment gets larger in com- 
parison to the spread of connections (therefore, to the 
size of the activity peak) , the capacity decreases approx- 
imately as 



-l/log(am) 



(47) 
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where am is the map sparsity and it is equal to: 



\M\ 



(48) 



where kd is a factor roughly equal to ^ 4.5 for the 1-D 
model and ~ 3.6 for the 2-D model. 

That is, the sparser the coding, the less the capacity. 
This is, at first glance, in contrast with what is known 
from the theory of auto- associative networks, in which 
sparser representations usually lead to larger storage ca- 
pacities. 

For comparison, keeping the formalism of for 
threshold-linear networks with hebbian learning rule, en- 
coding memory patterns {ri}i^i,,,N with sparsity a de- 
fined as 



((r2)> 



(for binary patterns this is equal to the fraction of active 
units), and for small a, the capacity is given by 



(49) 



aplog{l/ap)' 



The apparent paradox (larger capacity with sparser 
patterns, smaller with sparser charts) is solved as one 
recognizes that each chart can be seen as a collection 
of configurations of activity relative to different points 



in space covering, as in a tiling, the whole environment. 
Each configuration is roughly equivalent to a pattern in 
the usual sense. Intuitively, and in sense that will made 
clearer below, a chart is equivalent, in terms of "use of 
synaptic resources" to a number proportional to a^ of 
patterns of sparsity am ■ 

The proportionality coefficient, or equivalently, the dis- 
tance at which different configurations are to be consid- 
ered to establish a correct analogy, will be dealt with in 
Appendix 

These considerations and the comparison of eq. ^ and 
eq. |9| make clear that ac is the exact analogue of the 
pattern autoassociators' ap. 

Figures 1, 2, 3, 4, approx. here 



C. Inhibition independent stability 

The dynamical stability of the solutions of eq. ^ is in 
general determined by the precise functional form chosen 
for the inhibition, which we assumed a function of the 
average overall activity in the net. Nevertheless, there 
are fluctuations modes which leave the average activity 
unaltered. Stability against these modes is therefore un- 
affected by the inhibition and may be checked already 
for a general model. Let us consider a "synchronous" 
dynamics, that is, all the neurons are updated simulta- 
neously at each time step. The evolution operator for the 
variables V{r, t) and p{t) is: 



V{r, t + gp{t)N 
p^{t + l)^g^a\M\p'mK^)) 



dr'^. ,. Vir\t) b(x(t)) 
m\M\ ^ ' p{t) ^ Pit) 
dr' 
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M\ ^ > Pit) + Pit) 



(50) 
(51) 



This evolution operator has as its fixed points Voir) = povoir) and po where voir) and po are the solutions of eq. 
|3^ , and i.e. the stable states of our system. 

We can linearize the evolution operator around (Vo(?'), po) and look for fluctuation modes (eigenvectors) ((5F(r), 5p) 
with 



drSVir) = 



M 



We obtain the following equations: 



X6Vir) = g^iuoir)) 
1 



dr'Kir - r')dVir') 



M 



gaiuoir))Sp 



XSp 



1 



M 



ga\M\{{K^)) druoir)voir) ] Sp + -ga\M\iiK^)) druoir)SVir 



(52) 

(53) 
(54) 



M 



where 



Uoir)^M-^ (^). 
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Inserting eq. M in eq. B2 



_ Im dr'^uoir)) drKjr - r')SVir')] ^^^^ 
fj^jdr<7{uoir)) 



Eq. q5| can be inserted again in eq. ^3|, obtaining a 
closed integral equation in SV. Unfortunately, this equa- 
tion is very difficult to solve, but we can derive a stability 
condition by making an ansatz on the form of the eigen- 
function SV{r). More precisely, let us concentrate on 
the 1-D case. We look for solutions with even symme- 
try (we know there must be an eigenfunction with odd 
symmetry, and eigenvalue equal to 1, corresponding to a 
coherent displacement of the activity peak). This kind 
of solution corresponds to spreading and shrinking of the 
activity peak. Let us assume that the even eigenfunction 
with the highest eigenvalue (the most unstable) has only 
two nodes (an even eigenfunction must have at least two 
nodes because of eq. |2|), at rg and — rg. Let us take 
the sign of the eigenfunction SV{r) such that SV{Q) > 0. 
From eqs. p2 and 55 we see that 



D. The fully connected model 

The treatment of the model with the fully connected 
network and a kernel K for connection weights satisfying 
the condition ( |3l| ) will use the replica trick to average 
over the disorder (the realizations of the ?^s) and will 
eventually lead to a non-linear integral equation for the 
average activity profile in the space of the "condensed 
map" very similar to eq. Let the Hamiltonian of the 
system be 



Sp<0. 



EE 



(59) 



Now, from eq. 



1 



A= i^l - ^ga\M\{{K^)) J ^^druo{r)vo{r) 

lga\M\{{K^)) [ dru,{r)^-^, 
2 Jm op 

and we recognize that 



d™o(r)^^ <0. 

M op 



Thus, 



with 



T ^ ga\M\{{K')) / druoir)voir). 

I M 



(57) 



(58) 



Thus, if the ansatz we formulated holds, we have a 
stability condition F > 0, which is found to be fulfilled 
for all the solutions we found relative to maximal storage 
capacity. This implies that the storage capacity result 
is not affected by instability of the solutions, provided 
of course that an appropriate form for inhibition is cho- 
sen. This stability result is also related to the correlation 
in the static noise for two solutions centered at different 
p.f.c.s, as we will show in App. ^ 

It can also be shown that by taking the a — > limit 
(i.e. the single chart case), one always has F > since it 
is VQ{r) = when uo{r) < 0. 



where now the J? are given by ( |24| ) with a generic kernel 



K{r-/) ^ k{r-r')- K 



(60) 



(56) where, again. 



K= {{Kir-r')). 



The free energy calculation is sketched in Appendix |^. 
Again, the stable states of the system are governe d by 
mean field equations. The mean field equation, eq. |C16| 
is an integral equation in the functional order parameter 
f (r), the average space profile of activity. 

If we are able to solve eq. |C16| and fi nd f °" (r) as a func- 
tion of w and g', by substituting eq. C17 and eq. C18 



in eq. Cll we have an equation that gives us the value 
of a corresponding to that pair {g' ,w). ac is then the 
maximum of a over the possible values of {g' , w). 

To solve eq. C16, it is easy to verify that if z;(r) is a 
solution of 



v{r) = g'Af ( j dr'K{r- r')v{r') + w 



(61) 



with 



w = w — K dr v{r) 
Jm 



that is, the same equations as eqs. B2 and B3, then 



v{r) — / dr [_L(r — r) — L] v{r) 
Jm 
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is a solution of eq. C16. v can therefore be interpreted 



as the average activity profile, apart from a constant. 
Eq. ^1 can be solved with the same procedure used for 
eq. and the maximum value of a can be found by 
maximizing over g' and w. 

The results for 1-D and 2-D environment are depicted 
in fig. |l] and ^ (lower curves). As we may expect from 
pattern autoassociator theory, the capacity is much lower 
than for the diluted model, due to an increased interfer- 
ence between different charts.. As the sparsity a ~ 
gets smaller, the capacities of the two models get closer, 
both being proportional to i^^^pjjj- Reducing the spar- 
sity parameter of space representations has therefore the 
effect of minimizing the difference between nets with 
sparse and full connectivity. 



E. Sparser maps 

A possible extension of this treatment is inspired from 
the experimental finding that, in general, not all the cells 
have place cells in a given environment. Ref. |lTl e.g. re- 
ported that ~ 28 — 45% of pyramidal cells of CAl have a 
place field in a certain environment. We would like to see 
how this fact could affect the performance of the multi- 
chart auto-associator. It is then natural to introduce a 
new sparsity parameter, the chart sparsity indicating 
the fraction of cells which participate in a chart. We 
will show that, for the capacity calculation, "sparse" 
charts are equivalent to a single "full" chart, of size . 
We will present the argument for the diluted case, the 
fully connected case is completely analogous. 

Let TO^ be equal to 1 if cell i participates in chart /z, 
that is with probability Qc- Thus, the synaptic coupling 
Jij will read 



acN 



(62) 



Let us consider a solution with one condensed map: 
cells participating with p.f.c. in r in that map will have 
a space related signal-to-noise ratio 



;(f) = gAf 



J dr'K{r 



Mr') 



(63) 



for all the neurons not participating in the condensed 
map we will have 



V 



(64) 



The noise will have a variance 



p-^aa^ylLlll] {{K\r-r^)), (65) 



that is Oc times the value we would get for the same 
number of "full" charts with size \M\/ar. and now 
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{''"ImWI^ (^y"dr'if(r-r')t;(r')+u;j + 
il~ac)Miw)}. (66) 



By comparing eq. ^ and eq. and remembering that 
for r far from the activity peak, v{f) ~ M{w) we real- 
ize that this y value is approximately equivalent to the y 
value we would get for "full" charts of size \M\/ac- 

Inserting eq. |6^ and eq. |6^ in eq. ^ one finds, for the 
maximal capacity: 



(sparse charts) 



aclog(^) 



(67) 



As we anticipated one may interpret this result as fol- 
lows: the capacity is the same as if we had taken 
"sparse" charts, including ^ N cells, and put them side 
by side to form one single "full" chart. If we have started 
with aC "sparse" charts we now have have OcQfC "full 
charts" . From eq. we see that we can store at most 



(full charts) 



C full charts and 



(full charts) 



log(^) 



and this explains eq. Therefore, this network is as ef- 
ficient in terms of spatial information storage as the one 
operating with full charts. 



IV. INFORMATION STORAGE 

Like a pattern auto-associator, the chart auto- 
associator is an information storing network. The cog- 
nitive role of such a module could be to provide a spatial 
context to information of non-spatial nature contained 
in other modules, which connect with the multi-chart 
module. Each chart represents a different spatial or- 
ganization, possibly related to a different environmen- 
tal/behavioral condition. Within each chart, a cell is 
bound to a particular position in space, thus being the 
means for attaching some piece of knowledge to a par- 
ticular point in space, through inter-module connections. 
To give a very extreme, unrealistic, but perhaps useful, 
example, let us assume that each cell encodes a partic- 
ular discrete item, or the memory of some events hap- 
pened somewhere in the environment, as in a "grand- 
mother cell" fashion, encoding "the grandmother sitting 
in the armchair in the dining room" . The encoding of 
the "grandmother" may be accomplished by some set of 
afferents from other modules. The multi-chart associator 
can then attach a spatial location to that memory of the 
"grandmother". The spatial location encoded is ideally 
represented for each cell by its p.f.c. 

In this sense, the information encoded in the network, 
which can be extracted by measures of the activity of the 
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units, is the information about the spatial tuning of the 
units, that is their p.f.c.s. 

To restate this concept in a formal way, we look for 



(68) 



that is the information per synapse that can be extracted 
from S different observations of activity of the cells with 
the animal is in S different positions , and the system 
in activity states related to chart /i. This quantity does 
not diverge as S oo, since repeated observations of 
activity with the animal in nearby positions do not yield 
independent information, because of correlations between 
activity configurations, correlations which decrease with 
the distance at which the configurations are sampled. 

The full calculation of this quantity involves a func- 
tional integration over the distribution of noise affecting 
cell activity as the animal is moving, and exploring the 
whole environment. In Appendix ^ we suggest a proce- 
dure to approximate this quantity based on an "informa- 
tion correlation length" such that samples correspond- 
ing to animal positions at a distance Ij yield approxi- 
mately independent information. 

Is is the amount of spatial information which is stored 
in the module. It is the exact analogue of the stored in- 
formation for pattern auto-associators 1^ . As for storage 
capacity, it is to be found numerically, by maximization 
over w and g. 

As for the capacity one can find the solution which 
maximizes Is ■ The resulting /max is a function of the size 
of the relative spread of connections a = 1/\M\, and it 
amounts to a fraction of bit per synapse (see fig. ||) . 

figure 5 approx. here 

As with pattern auto-associators, the information 
stored increases with sparser representations. The in- 
crease is more marked for the fully connected network. 
For very sparse representations the performance of the 
fully connected model approaches the extreme dilution 
limit. 



V. DISCUSSION 

We have studied the multi-chart threshold linear as- 
sociator, as a spatial information encoding and storage 
module. We have given the solution for the dot-product 
kernel model, then we have introduced a formalism in 
which the generic kernel problem is soluble. 

The second treatment has the advantage of providing 
a form for the average activity peak profile, which can be 
compared with the experimental data (see for example 
@, fig-1). 

We have shown that the non-linear integral mean field 
equation (eq. |4|) can be solved at least for one class of 
connection kernels K{r ^ r'). 



The storage capacity for both models has been found. 
We note that the capacity for the dot-product model is 
compatible with the wide kernel (non sparse) limit of the 
generic model in one and two dimensions in the fully con- 
nected and in the diluted condition. 

The generic kernel treatment makes it possible to ma- 
nipulate the most relevant parameter for storage effi- 
ciency, i.e. the spread of connections. It is shown that 
this parameter plays a very similar role as sparsity for 
pattern auto-associators. In the multi-chart case, more- 
over, the effective sparsity of the stable configurations is 
determined also by the value of the gain parameter g, 
as shown analytically for the noiseless case. Neverthe- 
less, the capacity of the network depends on the spread 
of connection parameter Oc = kd/\M\ through a relation 
which is the exact analogue of the relation between spar- 
sity and capacity for the pattern auto-associator, at least 
in the very sparse limit. 

We have only considered here the capacity problem for 
one form of the connection kernel, although the treat- 
ment we propose is applicable, at least, to the other ker- 
nels considered for the noiseless case. Our hypothesis is 
that a similar law for sparsity is to be found as eq. 47, at 



least in the high sparsity limit, for more general forms of 
the kernel. 

We have then shown that the capacity scales in such 
a way that the information stored is not changed when 
only a fraction of the cells participate in each chart. In 
this case the firing of a cell carries information not only 
about the position of its p.f.c. in the chart environment, 
but also about which environment the cell has a place- 
field in. This information adds up, so that l/oc charts 
can be assembled in a single larger chart of size l/oc 
times larger. 

We have introduced a definition of stored information 
for the multi-chart memory network, which measures the 
number of effective different locations which can be dis- 
criminated by such a net: representations of places at a 
distance less than Ij are confused, because of the finite 
width of the activity peaks, and because of the static 
noise. 

1 1 does not vary much when \M\ varies. This is consis- 
tent with the fact that the storage capacity is well fitted 
by eq. ^ with kd 4.5. turns out to be ~ 3.5 for 
the 1-D model, with the arbitrary value for / of 0.95. // 
is therefore similar to the "radius" of the activity peak 
which should correspond to the "pattern" in the parallel 
between the chart auto-associator and the pattern auto- 
associator. 

It was not possible to carry over the calculation of ri2 
and I2 in the 2-D model as it turns out to be too compu- 
tationally demanding. Therefore we are not able to show 
the values of the storable information The fact that the 
storage capacity follows eq. ^ also in this case is an in- 
direct hint of a behavior very similar to what is found in 
1-D. 
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Appendix 



FREE ENERGY FOR THE 
"DOT PRODUCT" KERNEL MODEL 



APPENDIX A: REPLICA SYMMETRIC 



The replica symmetry free-energy reads 



f = -tU[ Dz\nTv{h, h2)\\ - - XI Ix^^^^'P - B{x) - ^ds^^^'V + s^"^ x^"^ - ^ t^^^^'x^^^'' - tx (Al) 

^ ^ ' ' (cr)J (cr)J ((t),1 

-nm + riyi + ^ ( ln[l - ro/3(yo - Vi)] - ^r^r T 

2p V 1 - Top(yo - yi) 



very much like eq.l9 in ||6| and with the same mean- 
ing for symbols, except that the population vector x'^'^^^' 
plays the role of the overlap x"^ , the vector Lagrange mul- 
tiplier t*^"^)'' appears instead of its scalar counterpart t" 
and the dimensionality d appears multiplying the last 
term, h and /i2 are 

h = -t-J2 t^"-*'' • ^"^'^ - 2(-2rri)i/2 (A3) 

ia),l 

h2=n-ro. (A4) 

((. . .)) means averaging over the distribution of p.f.c.'s 
ff. T is the noise level in the thermodynamic analysis. Tq 
is defined here as: 



(tl.r/('^)'')(4.r/('^)-') 



(A5) 



and it is found to be equal to 1/2 in 1-D and to 1 for the 
2-D torus. 

The saddle point equations can be found from this 
equations, and t and t'^'^^'' can be eliminated, in the same 



way as in |^]. Carrying on the calculation the T — 
equations eventually reduce to two equations in the two 
variables (in the case of a single "condensed" map): 



p 

(x' + sO 



(A6) 
(A7) 



Take for simplicity |v'| = v (while the direction is set 
by v' cx s'). The two equations read: 



El {w, v) = {Al + SA2f - a^3 = 

E2iw,v) EE 

1 



{Al + SA2 



9To{l + S) 



(A8) 

a- A2] - aA2 = (A9) 



where 5 = |s^|/|x^| is the relative importance of the 
external field and: 



^iK^^) = ^ / /V . ^ p Dz{w + Y,w' ■rf-z)\\- (J^f 



A2{w,v) 
A^{w,v) 



//fDz{w + Y^.^-rf-zr\\ 



Dz 



(AlO) 
(All) 
(A12) 



Dz is the Gaussian measure 

(2^)-i/2e-V2rf^. The + sign on the integral means that integration extremes are chosen 
such that {w + ^iV^ ■ ff — z) > 0. 

When the quenched average on the 77's is performed, Ai, A2, A3 reduce to (for the d-dimensional torus C^): 



[{w + v^ cos 6^ - vTo)^{w -f t; X cos 6*') + (w + D X cos e')a{w + d ^ cos 6*')] 



(A13) 
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[{w + v'^ cos 6'')$(w + w ^ cos 6*') + (w + w ^ cos 6^)a(w + w ^ cos 6^ 



(AM) 



^3(w,w) 



[1 + (w + u^cos6'')^]$(w + i;^cos6'') + (w + i; ^ cos 6l')cr(w + « cos 6>' 



(A15) 



where 



$(x) 
cr(a;) 



/27r 



=e 2 



(A16) 
(A17) 



APPENDIX B: GENERIC KERNEL, EXTREME 
DILUTION 

Let us consider the one-dimensional case first, and con- 
sider the kernel 



K{r- r') ^ k{r- ^) - e-'"'! - 

Eq. ^ can be written 

v(r) = gN (^j drk{r~ r)v{r ) + w 

where 



(B2) 



w — w — 



(B3) 



For the purpose of finding ac, maximizing with respect 
to w is equivalent to maximizing with respect to w. 
To solve eq.B2l the transformation 



V 9 

is used, which results in 

u[r) ^ 9 j dr'k{r~ f')N[u{f)] + w. 
By differentiating twice we get 



(B4) 



(B5) 



'(f) = -2gJ\f[u{r)] + u{f} - w = -—U[u{f)] (B6) 

du 



where 



U = 



du' {2gN{u') -u' + w) 



(B7) 



The differential equation (B6) is locally equivalent to 
the non- linear integral equation ( [B5| ) Equation (B6) must 
be solved numerically. As in the single ma p c ase, not 
all the solutions of the differential equation ( p6| ) are so- 
lution of the integral equation (B5). Solutions of (B£) 



are solution of (B5), strictly speaking, only in the case 
i\/ = R. Nevertheless, we force the equivalence since, 
also in the case of limited environments, with periodic 
boundary conditions, possible pathologies are not impor- 
tant for solutions with activity concentrated far from the 
boundaries. 



In order to classify the solutions of eq. B6 it is useful 



to study the "potential function" lA. If w is negative and 
large enough in absolute value, U{u) has a maximum and 
a minimum at the two roots of equation 



—U{u) = 2gN{u) - u w = 0, 
du 



or, in terms of v: 



V = gj\f{2v + w), 



(B8) 



(B9) 



corresponding to constant solutions of eq. ^ We look 
for solutions representing a single, symmetric peak of ac- 
tivity centered in r = 0. We therefore need to solve the 



Cauchy problem given by eq. B6 with the initial condi- 
tions: 



u(0) = uo 
u'(0) 



(BIO) 
(Bll) 



From fig. ^ it is clear that if uq > u* the solution will 
escape to — oo for r tending to infinity. This will corre- 
spond to V tending asymptotically to 0, and this solution 
cannot be a solution for the integral equation ( ^2[ ) as the 
asymptotic value must be a root of (|B9|). 

The solutions of the problem with uq < u* are pe- 
riodic, corresponding to multiple peaks of activity, and 
they are discarded as unstable with the same arguments 
holding for the single map case. There is also the con- 
stant solution 



u{r) 



(B12) 
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which obviously will not correspond to space related ac- 
tivity. The solution corresponding to the single activity 
peak can only be the one with uq = u* . It tends asymp- 
totically to Umax- This solution can be found numerically 
and inserted in eqji^ to find the value of a associated 
with the pair (.g,w). The solution will only be present 
for values of w for which U{u) has the extremal points 



and Umim that is: 



w < w 



(B13) 



where w* is equal to —2gAf{u*) + u* and u* is the root 
of the equation: 



2.9 



(B14) 



obt aine d by derivating twice lA^ and this shows that 
eq. B5 cannot have solutions for g < 1/2, as in the single 



map case. 

In the two dimensional case, we can consider the kernel 



K{r — r) — K{r — r) 



2 

W\ 



where K is the kernel having Fourier transform: 

2 



if(p) = 



1-Hp" 



(B15) 



(B16) 



The solution is worked out in the same way with the 
transformation (B4) and application of Laplacian. If we 



consider solutions with circular symmetry and pass to 
polar coordinates (r, (j)) , the equation for r reads: 



u"{r) + ^u'{r) = ~2gN[u[r)\ + u{r) - w 



(B17) 



We still have a single peak solution with tends asymptot- 
ically to Umax, but in this case we cannot rely on the lA 
function argument to find the initial condition at r = 0, 
which has to be found numerically. 
Figure 6 approx. here 



APPENDIX C: REPLICA FREE ENERGY 
CALCULATION FOR THE GENERIC KERNEL 

Again we will consider an environment M with peri- 
odic boundary conditions. We assume that there exists 
a kernel L such that 



dr"L{r~ r")L{r" - r') = \M\K{r- r'). (CI) 



Instead of the vector order parameter x'' we used for 
the dot-product kernel case (or of the scalar overlap x'^ 
of 1^) we can use the functional order parameter 



(C2) 



in terms of which the interaction part of the Hamiltonian 
@ reads 



2 ■ ^ 

^ E E E [^(-t - ^7) - K] v.v, - ^mo) - K) 

i j i 



(C3) 



Introducing the "square root" kernel L allows us to perform the standard Gaussian transformation manipulation and 
to carry out the mean field free energy calculation in the replica symmetry approximation: 

/ = -^((/ ^^lnTr(/i,/i2)^^ jjr[x^r)f - ^(^(0) - K)y, + B{x) - 



royo + riyi + 

I M 

$i:(Mi-wfa„-„oi- ,_^„(^X-») ) <^*' 

where now T'o(p) is the Fourier transform of the kernel |Af |if 

To(p) = |M| / dre-''P^K{r). (C5) 

J M 
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We now have 



h = h(x) + V / d^x'^ij') {L(f - r') - Z] - z(-2tri)i/2 
hi = -ro + ri. 

The T = mean field equations are much hke in Q apart from the x" (f) equation which reads 
x^if) = g' i^L{r" ~-r)-L\ Dz | j dr' [L{t^ -r)-L\ x''{r') + h{x) - 9 - p. 
where now the + sign on the integral means that the limits of integration over z are chosen such that 

dr' [L{r" -r')-L\ x''{r') + b{x) - 9 > 0. 



g' is a renormalized gain, which takes into account the effect of static noise, defined by: 



where ^ is given by eq. C18. 

The noise variance is given by 



-2Tr^ = a 



E 



where 



yo = [g'f 



Dz\ I dr' [L{r^ -r')- L\x''{r') + b{x) 

IM 



and 



We now pass to the rescaled variables 



^ = 9' 



v^ir) 



w 



Dz 



x-^ir) 
P 

b{x) 



P 



obtaining 



p 



'( dT^[L{T^ -r)-L]Af(w+ f dr' [L{r^ -f*)- L]v''{r) 
Jm \ Jm 



(C6) 
(C7) 

(C8) 
(C9) 
(CIO) 



(Cll) 

(C12) 

(C13) 

(CM) 
(C15) 

(C16) 
(C17) 
(C18) 
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APPENDIX D: GENERIC KERNEL: 
STORABLE INFORMATION CALCULATION 

First, the information per synapse we get from a sin- 
gle observation of activity, with the animal in a certain 
position times the number of stored charts is 

h = 



dr 

W\ 



'2-K 



2 log 



e 2 



J 



dr' 

w\ 



[1 - 0(u(r))] log 



[1 - c^{u{r))] 



(Dl) 



Next, we wish to calculate the joint information from 
two measures of activity, from the same cells, from all 
charts, while the rat is in two different locations, at a 
distance e. These two measures are correlated random 
variables: let 

Vi - [hi - pz{\ + 

be the activity of a cell measured while the rat is in po- 
sition 1, and 

V2 = - PZ2\^ 



be the activity of the same cell while the rat is in position 
2. 

The two noise variables are distributed according to a 
joint bivariate gaussian distribution: 



p{zi, Z2) 
1 



27rv/l - 4 



: exp 



2(1 -rf^) 



{zi ^zi- 2ri2ZiZ2) . (D2) 



The correlation coefficient ryi is a function of the dis- 
tance e, implicitly defined through the equation 



p2ri2(e) = a|M|((if2))y^,(,) 
where ?/i2is defined as 



(D3) 



V\2\ 



and assuming periodic boundary conditions: 



(D4) 



2 „2 



2/12 (e) = p g 



dr 

m 



++ 



Dzi2 X 



dr' 

W\ 



K{r — r')v(r') + w — zi 



dr^ 

w 



K{r - r")v{r" + e) + w ~ Z2 



(D5) 
(D6) 



or, 



2 2 f dr [++ 



u{r)u{r + e) (D7) 



where u{r) is defined by eq. B4. The integration measure 
for the noise variable is defined as 



is a solution, but now consider what happens when e in- 
creases. 

Figure 7 approx. here 

The derivatives of 



Dzi2^ I dzidz2p{zi,Z2). V{ri2,() = a{{K^))g^ I —Q{u{r),u{r + e),ri2)-ri2 

/M(r)-zi>0,ti(r+e)-Z2>0 •' I-*" I 



dr 



(D8) 



(DIO) 



Inserting eq. D7 in eq. P3| we yield: 

r,2 = a\M\{{K^))g^ J drQiuir),u{r + e),r,2) (D9) 
where 

.++ 

Q{x,y,ri2) = / Dzi2{x ~ zi){y " Z2). 



with respect to e and r'12 must be taken into considera- 
tion. One has: 

|p(.,2 = M = 0)=a((XW/^x 



d_ 
dy 



Qix = u{r), y^u{r + e), l)u'(r) = 0, (Dll) 



Eq. D9 can be solved numerically, an example is pro- 
vided in fig. 0, but a few features can be explored ana- 
lytically, in the neighborhood of e = 0. ri2 = l,e = 



P(ri2 = 1,6 = 0) <0 



(D12) 
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and 



dr 



12 



-V{ri2 -> l,e = 0) 



dr 



(D13) 



Fro m eg . D12 it turns out that when the derivative in 
eq. D13 is greater than zero, the solution ri2 = 1 disap- 



pears as one moves from e = 1, but another solution is 
still present so that 



lim r 12(e) < 1. 



(D14) 



Note that the condition 
d 



dry. 



is equivalent to 

r = ga\M\{{K^)) I dr u{r)v{r) < 0, 



(D15) 



M 



and the quantity F enters in t he st ability analysis con- 
siderations we sketched in sec. IIIC, at least for the 1-D 



case. Solutions with F > are stable against inhibition 
orthogonal fluctuations, so t hat it is likely that the possi- 
ble pathology implied by eq. D14 reflects an instability of 



the solution. We have always found numerically that for 
solution corresponding to the maximal storage capacity 
and information, F > 0. 

Once we know the joint probability distribution for zi 
and Z2, we can calculate the information we can extract 
about the p.f.c. of a cell from two measurements of activ- 
ity, while the rat is standing in two positions at a distance 
e, from all charts. 



/2(e) = a 




++ 



DZ12 



2(1- 



-2— (zj + Z2- 2ri2ZiZ2) 
'"12J 



log 



M 



dr' , 1 



\{u{r') - u{r) + zif + {u{r' + e) - u{r + e) + 23)^ 



12; 



-2ri2(u(r') - u{r) + zi)(M(r' + e) - ^(r + e) + Z2)])) 



+2 



Dz,2 
dr' 



log 



z'2>u{r+e) 

dz'2 exp 



dz'2 , 1 



12 



-(z2 + 42_2ri22i4))) 



'^°^0M|M|4>„(.,+,f-^'^^^ 2(1 -r\,) 
'2rx2(u(r') - u(r) + zx){u{r' + e) - u{r + e) + Zj)])) 



\(u(r') - u(r) + zif + {u{r' + e) - u{r + e) + Z2) 



' ^2 



log(/ 



dz'ldz'2 



[>u{r),z'2>u{r+f.) 



Dzi2 

dr' 



M 1^1 Jz[>u{r'),z:^>u(r'+e) 27r\/l - rj. 



1 



:exp( 



2tt^- 

exp(- 



-log( 

[(u(r') - u{r) + z'^f + + e) - %s.{r + e) + z'^f 
-2ri2(w(r') - u(r) + z'^){u{r' + e) - u{r + e) + z^)])) 



2(1 -r-?^) 
1 



(z^'-f z^'-2ri2z;4))) 



2(1 



' 12 



(D16) 



The minus signs (— ) beside the integration signs mean 
that respectively the first, or the second condition deter- 
mining the integration intervals in eq. DJ are reversed. 
The first term in the sum accounts for the contribution 
coming from measurement in which both activity values 
are positive. The second term is the contribution from 
measurements in which one value is zero and the other 
is positive. The third term comes from measurements in 
which both values are zero. 

For e = 0, /2 = -fi, since the two measures are identical. 

For large e one has I2 ^ 2/i, because the noise decor- 



relates and because in general the two measures will give 
non-zero results in distinct regions of the environment. 
The behavior of I2 as a function of e is exemplified in 
figure ||. We define as "information correlation length" 
the value // of e for which 



h = //i 



(D17) 



where / is a fixed fraction, say 0.95. Note that this quan- 
tity We may say that measurements of activity with the 
rat in two positions at a distance Ij give independent 
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information. 



Figure 8 approx. here 



This allows us to define as the stored information Ig 
the quantity 



that is, sampling the activity of a cell times, with the 

animal spanning a lattice with size li, we may effectively 

add up the information amounts we get from each single 
sample, as if they were independent. 
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10"' 10"' 10"' 

1/|Mj 

FIG. 1. The storage capacity plotted as a function of the 



"map sparsity" am, for the 1-D model, for the extremely di- 
luted (upper curve) and the fully connected (lower curve) 
limit. 




10"' 10-= 10-' 

1/|M| 

FIG. 2. Same as fig. ^ for the 2-D model. The capacity is 
smaller than for the 1-D model for the same Um- 
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FIG. 3. The "activity peak" profile corresponding to the 
solution of eq. ^ at the maximal storage level at \M\ — 30 
and \M\ = 15. The second case is plotted expanded to match 
the environment size of the first one and to show the effect of 
more widespread connections. 
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FIG. 4. The maximal storage activity peak profile in 2-D 
at \M\ = 400. 
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FIG. 5. The maximal storable information per synapse, as 
a function of 1/\M\. 
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FIG. 7. The r\2 function plotted as a function of the dis- 
tance between the two p.f.c.s, e in the \M\ = 30 case. 
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FIG. 6. The "potential" function U{u) defined by eq 



B7 



and entering the differential equation eq. B6, Solutions with 
u'(0) = and w(0) = mo, with Umax < uo < u* are oscillating. 
The solution with wo = u* is the one we seek, asymptotically 
approaching "Umax T > OO. 
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FIG. 8. The I2 function plotted as a function of the dis- 
tance between the two p.f.c.s, e in the \M\ = 30 case. Note 
that li, with / = 0.95 (see eq. D17) would be approximately 
3.5. This is seen not to change much when \M\ is varying 
(not shown). 
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